Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FAKE 5 #1281

Closed
wants to merge 215 commits into from
Closed

FAKE 5 #1281

wants to merge 215 commits into from

Conversation

matthid
Copy link
Member

@matthid matthid commented Jun 26, 2016

From #1232

Docs

https://github.com/matthid/FAKE/blob/coreclr/help/fake-dotnetcore.md

To test this branch

./build.sh

TODO (for alpha release):

  • bootstrap the build process
  • https://github.com/dotnet/coreclr/blob/master/Documentation/building/debugging-instructions.md -> Workaround implemented
  • Mono.Cecil (make it compile with latest changes (with netstandard1.5) jbevain/cecil#269)
  • Argu (Core clr support. Argu#57)
  • Make Fake work with dotnet run -f net46 (and remove everything but the core)
  • Make Fake run with dotnet run -f netcoreapp1.0
  • Things to cleanup
    • FCS.netcore: Chockes on checker.ParseAndCheckProject(options): internal error: Object reference not set to an instance of an object. (Workaround for now -> no fsi object in build script). Disappeared after latest FCS update (merge with visualfsharp)
    • Yaaf.FSharp.Scripting: How do we resolve FSharp.Core with sigdata and optdata in this new world? Required for let functionality. (Workaround for now -> no fsi object in build script)
    • Error **** error: internal error: The type initializer for '<StartupCode$FSharp-Compiler-Service-netcore>.$Reshapedmsbuild' threw an exception. when running with fsi object support -> not important in the first version
    • error FS0192: internal error: the IL instruction Microsoft.FSharp.Compiler.AbstractIL.IL+ILInstr+I_seqpoint cannot be emitted, See FSharp.Compiler.Service coreclr package is broken fsharp/fsharp-compiler-docs#591 and Properly embed 'FSIstrings' resource, fixes #591 fsharp/fsharp-compiler-docs#592
    • Remove hardcoded references in FSIHelper.fs
    • Remove hardcoded path in Yaaf.FSharp.Scripting
    • Remove binaries from lib folder (currently these "special" version are required) -> Can be done one by one once they become available.
  • Try to make it run standalone -> we can test the bootstrapping experience from this point on (but FAKE scripts cannot have dependencies)
  • Add the fake coreclr tool -> maybe bootstrapping is better? -> Later
  • Make Paket CoreCLR ready (Dotnetcore build for Paket.Core Paket#1785)
  • Add Paket.Core to FAKE, read the script, add the resolving step and add the #load lines
    • Restore after Update as well
    • Bug: Relative references via "paket.inline"
    • Make the caching layer in fake load the correct assembly (depending on the current runtime and the runtime folder of the packages) if necessary (Microsoft.NETCore.Platforms) -> Not a showstopper for now as for Fake-Packages runtime- and compile-time assemblies are the same. We fallback to already loaded assemblies anyway (therefore we can load netstandard for now)
  • Move the new "Core" to "Fake.Runtime"
    • Remove duplicate code from FakeLib (mutables!) Will be done on demand.
    • Remove Conditional Compilation from LegacyApi (was the first approach)
    • Remove StringHelpers from Fake.Runtime
    • Remove BuildServerHelper from Fake.Runtime (the interface probably need to remain, but it should be possible to extend)
    • Strip FSIHelper.
  • Bootstrapping
  • Testing
    • Add dotnetcore build to AppVeyor
    • and Travis
    • Add integration tests for the dotnetcore build
  • 'New' Library / Fake Components (a lot of packages are out of scope for this PR, see FAKE Dotnetcore #1232)
    • Fake.Core.Environment
    • Fake.Core.String
    • Fake.Core.BuildServer
    • Fake.Core.Tracing
    • Fake.Core.Targets
    • Fake.Core.Tasks
    • Fake.DotNet.MsBuild
    • Fake.IO.FileSystem
    • Fake.Core.Globbing
    • Fake.Core.Process
    • Fake.DotNet.Cli
    • Fake.Testing.Common
    • Fake.DotNet.Testing.MSpec
    • Fake.DotNet.Testing.NUnit
    • Fake.DotNet.Testing.XUnit2
    • Fake.Core.SemVer
    • Fake.Windows.Chocolatey
    • Fake.Core.ReleaseNotes
    • Fake.DotNet.NUnit
    • Fake.BuildServer.AppVeyor -> Later
    • Fake.DotNet.FSharpFormatting -> Later
    • Fake.DotNet.AssemblyInfo
    • Fake.Tracing.NAntXml
  • Fix Publish on preview4.
  • Future: Trim packages (here and here), see FAKE Dotnetcore #1232

@matthid matthid mentioned this pull request Jun 26, 2016
@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

@forki FYI

$ ./build.sh Default

Starting download from https://github.com/fsprojects/Paket/releases/download/3.3.2/paket.exe

Done.

Paket version 3.3.2.0

Paket failed with:

    Error in paket.dependencies line 33

  Package xunit.runners is defined more than once in group Main of /home/travis/build/fsharp/FAKE/paket.dependencies

@cloudRoutine
Copy link
Member

if you're still having build issues try it with these settings in your project.json

{
  "name": "",
  "version": "1.0.0-*",
  "buildOptions": {
    "compilerName": "fsc",
    "compile": {
      "includeFiles": [

      ]
    }
  },
  "dependencies": {
    "Microsoft.FSharp.Core.netcore": "1.0.0-alpha-160509",
    "FSharp.Compiler.Service.netcore": "1.0.0-alpha-00001",
    "FSharp.Compiler.Service.ProjectCracker.netcore": "1.0.0-alpha-00001",
    "NETStandard.Library": "1.5.0-rc2-24027"
  },
  "tools": {
    "dotnet-compile-fsc": {
      "version": "1.0.0-*",
      "imports": [
        "dnxcore50",
        "portable-net45+win81",
        "netstandard1.3"
      ]
    }
  },
  "frameworks": {
    "netstandard1.5": {
      "imports": [
        "portable-net45+win8",
        "dnxcore50"
      ]
    }
  }
}

and these in your nuget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <!--To inherit the global NuGet package sources remove the <clear/> line below -->
    <clear />
    <add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />
    <add key="dotnet-cli" value="https://dotnet.myget.org/F/dotnet-cli/api/v3/index.json" />
    <add key="cli-deps" value="https://dotnet.myget.org/F/cli-deps/api/v3/index.json"/>   
    <add key="fsharp-daily" value="https://www.myget.org/F/fsharp-daily/api/v3/index.json" />    
    <add key="fsharp-compiler-service" value="https://ci.appveyor.com/nuget/fsgit-fsharp-compiler-service"/>
    <add key="api.nuget.org" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>

@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

@cloudRoutine Build is fine net46 and netcoreapp1.0 (the AppVeyor/Travis issue is unrelated). Running in net46 works as well. Only running Fake.netcore with netcoreapp1.0 is broken. I think I have most of the stuff already added to project.json and NuGet.config, but I will look again later, thanks.

@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

Updated first post with the current errors I'm encountering...
Seems like I build FSharp.Compiler.Service wrong?
IIRC I did just build DotnetCliBuild and then copied the resulting nuget package FSharp.Compiler.Service.netcore.1.0.0-alpha-00001.nupkg to this branch.

@cloudRoutine I switched to the myget feed which contains FSharp.Compiler.Service (and deleted local nuget caches). Sadly the same thing happens. Thanks again for the tipp.

@cloudRoutine
Copy link
Member

cloudRoutine commented Jun 26, 2016

@matthid I managed to get it to run, but I had to remove some dependencies and functionality. You'll need to find compatible versions to work with what's here or try another implementation approach.

Fake.netcore
Fakelib.netcore

It's a pain to do, but you can usually find the sources of conflict in the project.lock.json

@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

@cloudRoutine with dotnet version are you using?

$ dotnet --version
1.0.0-preview1-002702

It's already working for me (at least after running dotnet publish, see first post). It just seems there are some bugs in FSharp.Compiler.Service.netcore

@cloudRoutine
Copy link
Member

dotnet --version
1.0.0-preview3-003161

i guess you're on top of it then 👍

@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

I have no idea. This whole versioning is so confusing. I have no idea if I should upgrade and try do make it work with preview3 or stay on preview1 which is suggested here: https://github.com/dotnet/netcorecli-fsc/wiki

@cloudRoutine
Copy link
Member

Honestly I'd wait until everything becomes more stable. There's no huge gain in rushing to try to get it done early when it could be done later much faster with less headaches.

@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

Wow... Now I get some internal F# compiler error: error FS0192: internal error: the IL instruction Microsoft.FSharp.Compiler.AbstractIL.IL+ILInstr+I_seqpoint cannot be emitted.

I think I have to give up for now.

@cloudRoutine Yeah I kind of agree for the cli tooling, that's why I have given up on the run -f netcoreapp1.0 thing. But as it "works" after publishing and the runtime should be "stable" now I think it is the time to make FAKE and the FSharp.Compiler.Service ready :).

Most of the errors now are probably FCS specific bugs and I'm not sure anyone has interest in fixing them (if not FAKE).

@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

Related: fsharp/fsharp-compiler-docs#593 (you need those changes for this branch)

@matthid
Copy link
Member Author

matthid commented Jun 26, 2016

Wow I made it work:

$ cat test.fsx
printfn "Hello FAKE"
$ dotnet exec bin/Debug/netcoreapp1.0/publish/Fake.netcore.dll run --verbose test.fsx
FakePath: C:\PROJ\FAKE\src\app\Fake.netcore\bin\Debug\netcoreapp1.0\publish\FakeLib.netcore.dll
FAKE - F# Make "4.29.2"
Running Buildscript: test.fsx
Cache doesn't exist
FAKE: Trying to resolve FSharp.Compiler.Service.netcore.resources, Version=1.0.0.0, Culture=en-US, PublicKeyToken=null
FAKE: Trying to resolve FSharp.Compiler.Service.netcore.resources, Version=1.0.0.0, Culture=en, PublicKeyToken=null
FAKE: Trying to resolve System.Core, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
FAKE: Trying to resolve FSharp.Compiler.Service.netcore.resources, Version=1.0.0.0, Culture=en-US, PublicKeyToken=null
FAKE: Trying to resolve FSharp.Compiler.Service.netcore.resources, Version=1.0.0.0, Culture=en, PublicKeyToken=null
Hello FAKE
Caching is disabled on CoreCLR, see https://github.com/dotnet/coreclr/issues/919#issuecomment-219212910
Ready.

@matthid
Copy link
Member Author

matthid commented Jul 2, 2016

Updated first post, updated to latest preview 2 tooling and changed to Argu for command line parsing.

@matthid
Copy link
Member Author

matthid commented Jul 3, 2016

I made a first release for win7 to check how bootstrapping could look like: https://github.com/matthid/FAKE/releases/tag/v1.0-alpha-01

@matthid
Copy link
Member Author

matthid commented Jul 9, 2016

How I'm moving this forward and my suggested migration path:

  • Creating a new library "FakeLib.Core" which will be the absolute core of what we need to run scripts
  • Move existing functionality one by one to new packages and make them reference each other and "FakeLib.Core" (if they need it). The new packages should be usable as libraries if possible (from regular Programs).
  • The existing "FakeLib" should reference all the new packages and "FakeLib.Core". This way we can continue delivering the known "Fake" (the package will just contain more files). We just need to make sure that #r "FakeLib.dll" still loads all files. (needs to be tested, if not we can create the "old" FakeLib by using ILMerge)
  • Once everything is converted we could in theory get rid of "FakeLib" and write a new console application using the new packages, while providing the "old" command-line interface. This would print a warning to the user to update Fake to the dotnetcore version.

This is the general Roadmap. For now I'm concentrating on the bootstrap experience and to get the most important features converted to separate packages.

The road so far:

  • Released https://github.com/matthid/FAKE/releases/tag/core-v1.0-alpha-02

  • Run simple scripts without dependencies

  • Fake is now capable of restoring dependencies before running the build script. You can specify dependencies within your build script in a Fake Dependencies section.

  • Either embed your dependencies, like:

    (* -- Fake Dependencies paket-inline
    source https://nuget.org/api/v2
    
    nuget System.Xml.XmlDocument ~> 4.0
    nuget System.Security.Cryptography.Algorithms ~> 4.0
    nuget Microsoft.FSharp.Core.netcore
    -- Fake Dependencies -- *)
    #load "./.fake/unusedDependencies.fsx/loadDependencies.fsx"
    
    open System.Xml
    
    printfn "Starting Build."
    
  • Or use an external paket.dependencies file:

    (* -- Fake Dependencies paket.depedencies
    file ./paket.dependencies
    group Main
    -- Fake Dependencies -- *)
    #load "./.fake/use_external_dependencies.fsx/loadDependencies.fsx"
    
    
    printfn "Starting Build."
    

    Note the typo "paket.depedencies" in the header. This will be fixed in the next version. Note also the "./" in the file specifier, this is currently required as well.

  • The command line is using Argu 3 and SubCommands (fake.netcore.exe run is for running scripts, just look into the --help output). I'm thinking about adding more sub-commands in the future.

  • The loadDependencies.fsx file is currently empty. It will be used to provide nice editor support for editing scripts in the future (it is not required/use by fake).

@smoothdeveloper
Copy link
Contributor

Somehow related a set of helpers to run dotnet.exe commands from FAKE:

https://github.com/dolly22/fake.dotnet

I'm also having a wild idea: since there are so many FAKE extensions (in separate packages), maybe we could study a ExtensionTypeProvider which takes care of downloading the package and exposing the helpers to the FAKE Script?

@matthid matthid changed the title WIP: CoreCLR support WIP: Dotnetcore support Jul 17, 2016
@matthid
Copy link
Member Author

matthid commented Apr 29, 2017

@matthid
Copy link
Member Author

matthid commented Apr 29, 2017

Its incredible what changes were necessary throughout the ecosystem to make this possible. When I started it I thought we just integrate paket and things would work :)

After almost a year this is finally green. This means we can finally look at FAKE instead of fixing all the other stuff.

This also means I will now work on branches on top of this and then merge them back here for the time being. Sadly this is less visible for others until we have the access rights figured out.

@matthid matthid changed the title Dotnetcore support FAKE 5 Apr 29, 2017
@matthid
Copy link
Member Author

matthid commented May 19, 2017

#1559

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants